Introduction

Hello, this is Hemanth from the Alliance Department. In this blog, I will demonstrate how to filter messages based on payload using Amazon Simple Notification Service (Amazon SNS), which now supports payload-based message filtering. This expands its feature set, which already included attribute-based message filtering.

AWS

Is a secure cloud service platform that offers compute power, database storage, content delivery, network, and other functionality to help businesses scale and grow. It is one of the first cloud vendors to start services in the year 2006. It offers all the 3 service models namely IAAS, PAAS, and SAAS. Some of the notable domains in AWS are Compute, Migration, Storage, Network and Content Delivery, Management Tools, Database, Messaging, Security and Identity Compliance, and many more.

S3

Simple and popular AWS Service for storage. Replicates data by default across multiple facilities. It charges per usage. It is deeply integrated with AWS Services. Buckets are logical storage units. Objects are data added to the bucket. S3 has a storage class on object level which can save money by moving less frequently accessed objects to a colder storage class.

SNS

A fully managed messaging service for both application to application and application to person. A2A pub/sub functionality provides topics for high-throughput, push-based, many-to-many messaging between distributed systems, microservices, and event-driven serverless applications. The A2P functionality enables you to send messages to users at scale via SMS, mobile push, and email.

Lambda

A serverless compute service that runs code as a reply to events and automatically takes care of the bottom resources. It runs code on high availability compute infrastructure and performs all the administration of the compute resources. A few examples are HTTP requests vis Amazon API Gateway, changes to objects in S3, and many others.

Demo

Open the AWS Management Console and search for S3. Click on Create Bucket. Provide a unique bucket name. Leave everything else as default and click on Create Bucket.


Bucket successfully created.

Go to the AWS Management Console and search for SNS. Click on Create Topic. Provide a topic name and click on Next Step.

Change the access policy of subscribers and publishers to Everyone, keeping everything else as default.

Click on Create Topic.

SNS topic succesfully created

Go to the AWS Management Console and search for Lambda. Click on Create Function. Provide a function name, choose runtime as Python, and click on Create Function.

Function successfully created. Make changes to the code and deploy the function.

In the Lambda function, click on Add Trigger.

Select SNS and choose the previously created SNS topic.

Go to the S3 bucket created earlier, click on Properties.

Scroll down to Event notifications and click on Create event notification. Provide an event name and tick the All objects create events checkbox.

Click on Save changes.

Create a folder in the S3 bucket, name it "filtered", and click on Create Folder.


Go to the SNS topic created earlier, click on the subscription created by Lambda.

Click on edit

Turn on the Subscription filter policy and change the code of the message body to filter messages based on specific criteria (e.g., anything uploaded with "image" in the name). Click on Save changes.

Successfully saved.

In the Lambda function, click on View CloudWatch Logs.

Check the log stream created by SNS when it was added to Lambda.

Now going back to S3 and in the filtered folder uploading image

Check CloudWatch logs to verify that Lambda was successfully triggered.

Checking inside the log

Check the metrics section in CloudWatch, under SNS, to see the invocation metrics.

Upload another object to the "filtered" folder to test the filtering with another name as below.

Check CloudWatch logs to verify that Lambda was triggered. This shows that lambda has not been triggered.

Check the metrics section in CloudWatch, under SNS, to see the invocation metrics and you can notice new entries.

Conclusion

Amazon Simple Notification Service (SNS) has improved its attribute-based filtering capabilities by adding payload-based message filtering functionality. By using subscription filter policies that assess message content, this enables more accurate and effective message management. This feature helps you minimize pointless processing, simplify event-driven systems, and make sure apps receive only pertinent data. This enhancement creates new opportunities for scalable communications operations and streamlines system design.